128 research outputs found
Empowering Dual-Encoder with Query Generator for Cross-Lingual Dense Retrieval
In monolingual dense retrieval, lots of works focus on how to distill
knowledge from cross-encoder re-ranker to dual-encoder retriever and these
methods achieve better performance due to the effectiveness of cross-encoder
re-ranker. However, we find that the performance of the cross-encoder re-ranker
is heavily influenced by the number of training samples and the quality of
negative samples, which is hard to obtain in the cross-lingual setting. In this
paper, we propose to use a query generator as the teacher in the cross-lingual
setting, which is less dependent on enough training samples and high-quality
negative samples. In addition to traditional knowledge distillation, we further
propose a novel enhancement method, which uses the query generator to help the
dual-encoder align queries from different languages, but does not need any
additional parallel sentences. The experimental results show that our method
outperforms the state-of-the-art methods on two benchmark datasets.Comment: EMNLP 2022 main conferenc
RUEL: Retrieval-Augmented User Representation with Edge Browser Logs for Sequential Recommendation
Online recommender systems (RS) aim to match user needs with the vast amount
of resources available on various platforms. A key challenge is to model user
preferences accurately under the condition of data sparsity. To address this
challenge, some methods have leveraged external user behavior data from
multiple platforms to enrich user representation. However, all of these methods
require a consistent user ID across platforms and ignore the information from
similar users. In this study, we propose RUEL, a novel retrieval-based
sequential recommender that can effectively incorporate external anonymous user
behavior data from Edge browser logs to enhance recommendation. We first
collect and preprocess a large volume of Edge browser logs over a one-year
period and link them to target entities that correspond to candidate items in
recommendation datasets. We then design a contrastive learning framework with a
momentum encoder and a memory bank to retrieve the most relevant and diverse
browsing sequences from the full browsing log based on the semantic similarity
between user representations. After retrieval, we apply an item-level attentive
selector to filter out noisy items and generate refined sequence embeddings for
the final predictor. RUEL is the first method that connects user browsing data
with typical recommendation datasets and can be generalized to various
recommendation scenarios and datasets. We conduct extensive experiments on four
real datasets for sequential recommendation tasks and demonstrate that RUEL
significantly outperforms state-of-the-art baselines. We also conduct ablation
studies and qualitative analysis to validate the effectiveness of each
component of RUEL and provide additional insights into our method.Comment: CIKM 2023 AD
Large Language Models are Diverse Role-Players for Summarization Evaluation
Text summarization has a wide range of applications in many scenarios. The
evaluation of the quality of the generated text is a complex problem. A big
challenge to language evaluation is that there is a clear divergence between
existing metrics and human evaluation. For example, the quality of a document
summary can be measured by human annotators from both objective aspects, such
as grammatical and semantic correctness, as well as subjective dimensions, such
as comprehensiveness, succinctness, and interestingness. Most of the automatic
evaluation methods like BLUE/ROUGE may be not able to capture the above
dimensions well. In this paper, we propose a new evaluation framework based on
LLMs, which provides a comprehensive evaluation framework by comparing
generated text and reference text from both objective and subjective aspects.
First, we propose to model objective and subjective dimensions of generated
text based on roleplayers prompting mechanism. Furthermore, we introduce a
context-based prompting mechanism that is able to generate dynamic roleplayer
profiles based on input context. Finally, we design a multi-roleplayer
prompting technology based on batch prompting to integrate multiple evaluation
results into evaluation results. Experimental results on two real datasets for
summarization show that our model is highly competitive and has a very high
consistency with human annotators
- …